Quantifying error in aerial survey data

Aerial survey, also referred to as aerial sketchmapping, is the technique of observing symptoms of forest damage from an aircraft and transferring the information manually onto a base map. Recent high levels of bark beetle mortality across the western United States have generated greater demands for, and more disparate uses of, aerial survey data. While aerial survey data are typically considered to be qualitative in nature, the recent application of the data has driven an interest in assessing its spatial and categorical accuracies quantitatively. This paper describes methods for assessing the accuracy of aerial survey data and discusses several implications and applications related to the error results. The error matrix and kappa (κ) statistic, commonly used to assess accuracies of image classifications in remote sensing, were used to describe errors present in the aerial survey data. Field crews collected ground data that were used to validate the aerial classifications on 233 plots across 17.3 million ha. An additional 24 plots were incorporated into the validation from a complementary project, bringing the total number of plots to 257. Errors within the aerial survey data were found to be acceptable for coarse-scale analyses but excessive for use at fine spatial scales. In addition, this paper discusses the benefits of error analysis which include the quantification of errors for reporting purposes, the inclusion of error rates in the metadata, and the ability to focus training programs and technology development by highlighting classes with the highest error. Finally, the cost associated with accuracy assessment implementation is described and weighed against the benefits.